Papers with AdamW optimizer
On the Hidden Objective Biases of Group-based Reinforcement Learning (2026.acl-short)
Copied to clipboard
| Challenge: | Recent studies have reported unexpected behaviors during training, including lengthrelated biases, formatting tokens, and reward hacking in multi-objective settings. |
| Approach: | They propose to analyze group-based reinforcement learning methods within a unified surrogate formulation. |
| Outcome: | The proposed methods exhibit structural mismatches between reward optimization and the underlying training objective. |
SAGE: Sign-Adaptive Gradient for Memory-Efficient LLM Optimization (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to train LLMs consume memory equivalent to twice the model size, resulting in a hybrid design that reverts to AdamW and negates the memory gains. |
| Approach: | They propose a new, memory-efficient O(d) adaptive scale that replaces AdamW in a hybrid structure that combines a Lion-style update direction with a memory-saving adaptive scale. |
| Outcome: | The proposed model outperforms existing methods on LLMs up to 1.3B parameters while significantly reducing optimizer state memory. |